Application No. 10/629,403 

Reply to Office Action dated: June 9, 2009 

Reply dated: October 8, 2009 

In the Claims: 

Please amend Claims 1, 10, 18, 24, 37 and 46; and cancel Claims 14-15, all as shown 
below. Applicant respectfully reserves the right to prosecute any originally presented claims in a 
continuing or future application. 

1. (Currently Amended) A method for exchanging information in a shared interactive 
environment, comprising: 

selecting a first remote physical device in a first live video image wherein the first remote 
physical device has information associated with it; 

causing the information to be transferred to a second remote physical device in a second 
live video image wherein the transfer is brought about by 

manipulating a visual representation of the information [[in]] by interacting with the 
first live video image and [[in]] the second live video imagej[;]] 

querying the second remote physical device to determine if it can receive the 
information from the first remote physical device, and 

transferring the information from the first remote physical device to the second 
remote physical device: 

wherein at least one of the first remote physical device and the second remote physical 
device has a statically or dynamically defined hotspot in the first live video image or the second live 
video image; 

wh e r ei n th e man i pu l at i on i nc l ud e s i nt e racting w i th th e f i rst li v e v i d e o i mag e and th e s e cond 
li v e v i d e o i mag e ; 

wherein the first remote physical device and the second remote physical device are part of 
the shared interactive environment; and 

wherein the first remote physical device and the second remote physical device are not th e 
same different remote physical devices . 

2. (Original) The method of claim 1 wherein: 

the manipulation is accomplished by dragging the representation from the first physical 
device and dropping the representation on the second physical device. 

3. (Original) The method of claim 1 wherein: 

changes to the physical devices are visible to all participants in the shared interactive 
environment. 
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4. (Original) The method of claim 1 wherein: 

a physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

5. (Original) The method of claim 1 , further comprising: 

annotating at least one of the first live video image and the second live video image. 

6. (Original) The method of claim 5, further comprising: 

automatically transferring the annotation to a physical device if the annotation is at least 
partially drawn over the physical device as it appears in a live video image. 

7. (Original) The method of claim 1 wherein: 

the first live video image and the second live video image are the same. 

8. (Original) The method of claim 1 wherein: 

the information can include a digital file, an annotation, a sound, and an audio/video 
presentation. 

9. (Original) The method of claim 1 wherein: 

at least one of the first physical device and the second physical device has associated with it 
a pop-up control panel through which a user can configure and control it. 

10. (Currently Amended) A method for exchanging information in a shared interactive 
environment, comprising: 

selecting a first object wherein the first object is one of 1) a physical device at a remote 
location shown in a first live video image, and 2) an icon on a computing device; 

causing information associated with the first object to be transferred to a second object 
wherein the second object is the other of 1) the physical device at the remote location shown in the 
first live video image, and 2) the icon on the computing device; 

annotating the first live video image; 

automatically transferring the annotation to the physical device if the annotation is at least 
partially drawn over the physical device as it appears in a live video image; 

displaying the annotation on the physical device such that the annotation can be viewed at 
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the remote location; 

wherein the first physical device has a dynamically defined hotpot in the first live video 

image; 

wherein the transfer is brought about by manipulating a visual representation of the 
information in the first live video image; 

wherein the manipulation includes interacting with the first object in the first live video image 
and the second object; and 

wherein the physical device is part of the shared interactive environment. 

1 1 . (Original) The method of claim 10 wherein: 

the manipulation is accomplished by dragging the representation from the first object and 
dropping the representation on the second object. 

12. (Original) The method of claim 10 wherein: 

changes to the physical device are visible to all participants in the shared interactive 
environment. 

13. (Original) The method of claim 1 0 wherein: 

the physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

14-15. (Canceled) 

16. (Original) The method of claim 1 0 wherein: 

the information can include a digital file, a sound, and an audio/video presentation. 

17. (Original) The method of claim 1 0 wherein: 

the physical device has associated with it a pop-up control panel through which a user can 
configure and control it. 

18. (Currently Amended) A method for annotating a live video image, comprising: 
annotating the live video image , wherein the live video image shows a location including one 

or more physical devices at the location ; 

automatically transferring the annotation to [[a]] one of the physical dov i co devices at the 
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location shown in the live video image if the annotation is at least partially drawn over the physical 
device as it appears in the live video image; 

displaying the annotation on the physical device such that the annotation can be viewed at 
the location; and 

wherein the annotation is visible to at least one participant in a shared interactive 
environment. 

19. (Original) The method of claim 1 8 wherein: 

the at least one participant can interact with the physical device. 

20. (Original) The method of claim 18 wherein: 

each of the at least one participants can interact with the shared interactive environment 
through different computing devices. 

21 . (Original) The method of claim 18 wherein: 

the live video image is one of: a panoramic view and a zoomed view. 

22. (Original) The method of claim 18 wherein: 

the physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

23. (Original) The method of claim 18 wherein: 

the physical device has associated with it a pop-up control panel through which a user can 
configure and control it. 

24. (Currently Amended) A shared interactive environment, comprising: 

a camera system to provide a first live view and a second live view different from the first live 
vie w of a location , wherein the second live view can be configured to zoom in on a portion of the 
first live view; 

a first graphical user interface (GUI) coupled to the camera system and to present the first 
live view and the second live view, wherein the views can capture a physical device; 

a device controller to dynamically control the physical device in response to interaction of a 
first user with the GUI wherein the interaction can i nc l ud i ng include annotating at least one of: 1) the 
first live view; and 2) the second live view; 
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wherein annotations are automatically transferred to the physical device in the live views if 
the annotation is at least partially drawn over the physical device as it appears in the live video 
image , and wherein the annotation is displayed on the physical device such that the annotation can 
be viewed at the location ; 

a device tracker coupled to the camera system and to dynamically recognize new physical 
devices; and 

wherein the camera system can be mounted on a mobile, robotic platform. 

25. (Original) The shared interactive environment of claim 24, wherein: 
the first GUI allows the first user to interact the physical device; and 

wherein the interaction of the first user is apparent to a second user via a second GUI. 

26. (Original) The shared interactive environment of claim 24, wherein: 

the device controller can control the physical device through at least one of: 1) an infrared 
communication channel; and 2) one or more networks. 

27. (Original) The shared interactive environment of claim 24, wherein: 

the device tracker can recognize new physical devices by at least one of: 1) image pattern 
recognition; 2) radio frequency transmission; and 3) acoustic signal. 

28. (Original) The shared interactive environment of claim 24, wherein: 
the physical device can be a display; and 

wherein the display can include an image stack. 

29. (Original) The shared interactive environment of claim 25, wherein: 

the first GUI can provide a second live view that is different from the second live view 
provided by the second GUI. 

30. (Original) The shared interactive environment of claim 24, wherein: 
the GUI is implemented as one or more web pages. 

31 . (Original) The shared interactive environment of claim 24, wherein: 

the first user can select the second live view by drawing a diagonal in the first live view. 



-6- 

Attorney Docket No.: FXPL-01063US0 
M:\nfeld\wp\FXPL\1063US0\1063US0_RespFOA_060909.doc 



Application No. 10/629,403 

Reply to Office Action dated: June 9, 2009 

Reply dated: October 8, 2009 

32. (Original) The shared interactive environment of claim 24, wherein: 

the physical device has a pop-up control panel that can be made apparent to the first user 
through the first GUI; and 

wherein the pop-up control panel allows the first user to control and configure the physical device. 

33. (Canceled) 

34. (Original) The shared interactive environment of claim 24, wherein: 

the physical device can be represented by a set of attributes and a set of behaviors. 

35. (Original) The shared interactive environment of claim 34, wherein: 
the representation of the physical device is part of a device hierarchy. 

36. (Canceled) 

37. (Currently Amended) A computer readable memory having instructions stored thereon that 
when executed by a processor cause a system to: 

select a first remote physical device in a first live video image wherein the first remote 
physical device has information associated with it; 

cause the information to be transferred to a second remote physical device in a second live 
video image wherein the transfer is brought about by 

manipulating a visual representation of the information [[in]] by interacting with the 
first live video image and [[in]] the second live video imagejf;]] 

querying the second remote physical device to determine if it can receive the 
information from the first remote physical device, and 

transferring the information from the first remote physical device to the second 
remote physical device; 

wherein at least one of the first remote physical device and the second remote physical 
device has a statically or dynamically defined hotspot in the first live video image or the second live 
video image; 

whoro i n tho man i pu l at i on i nc l udes i ntoract i ng w i th tho f i rst li vo v i doo i mago and tho second 
li vo v i doo i mago; 

wherein the first remote physical device and the second remote physical device are part of 
the shared interactive environment; and 
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wherein the first remote physical device and the second remote physical device are not th e 
same different remote physical devices . 

38. (Previously Presented) The computer readable memory of claim 37 wherein: 

the manipulation is accomplished by dragging the representation from the first physical 
device and dropping the representation on the second physical device. 

39. (Previously Presented) The computer readable memory of claim 37 wherein: 

changes to the physical devices are visible to all participants in the shared interactive 
environment. 

40. (Previously Presented) The computer readable memory of claim 37 wherein: 

a physical device can include a display, a projector, a printer, a facsimile machine, a 
personal digital assistant, a computer, and a portable computer. 

41. (Previously Presented) The computer readable memory of claim 37 further comprising 
instructions that when executed cause the system to: 

annotate at least one of the first live video image and the second live video image. 

42. (Previously Presented) The computer readable memory of claim 37 wherein: 
automatically transferring the annotation to a physical device if the annotation is at least 

partially drawn over the physical device as it appears in a live video image. 

43. (Previously Presented) The computer readable memory of claim 37 wherein: 
the first live video image and the second live video image are the same. 

44. (Previously Presented) The computer readable memory of claim 37 wherein: 

the information can include a digital file, an annotation, a sound, and an audio/video 
presentation. 

45. (Previously Presented) The computer readable memory of claim 37 wherein: 

at least one of the first physical device and the second physical device has associated with it 
a pop-up control panel through which a user can configure and control it. 
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46. (Currently Amended) A system, comprising: 

means for selecting a first remote physical device in a first live video image wherein the first 
remote physical device has information associated with it; 

means for causing the information to be transferred to a second remote physical device in a 
second live video image wherein the transfer is brought about by 

manipulating a visual representation of the information [[in]] by interacting with the 
first live video image and [[in]] the second live video image,, 

querying the second remote physical device to determine if it can receive the 
information from the first remote physical device, and 

transferring the information from the first remote physical device to the second 
remote physical device ; 

wherein at least one of the first remote physical device and the second remote physical 
device has a statically or dynamically defined hotspot in the first live video image or the second live 
video image; 

whoro i n tho man i pu l at i on i nc l udes i nteracting w i th the f i rst li ve v i deo i mago and the second 
li vo v i deo i mago; 

wherein the first remote physical device and the second remote physical device are part of 
the shared interactive environment; and 

wherein the first remote physical device and the second remote physical device are not th e 
same different remote physical devices . 
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